Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE] Added support for LocalAI Speech To Text configuration #2376

Merged
merged 2 commits into from
May 13, 2024

Conversation

clates
Copy link
Contributor

@clates clates commented May 9, 2024

Cleaned up and documented some of the constants and their requirements in SpeechToText.jsx - That'd be a great area to refactor in the future. More than a few places relied on the provider's name to match the key.

I made this a little more explicit by consolidating the SpeechToTextType, but the various accessors probably should just be looking at the key or the name and not both.


STILL NEEDED:

  • I am confident nothing broke with the OpenAI nor AssemblyAI implementations, but I don't have credentials to test manually and I didn't see a testing framework in place to assert against.

  • SpeechToText.jsx
    import localAISVG from '@/assets/images/robot.png'
    This is a random picture that was in the @/assets/images/ dir, change to a LocalAI image

  • The SpeechToTextType is duplicated in both ui/src/ui-component/extended/SpeechToText.jsx and in components/src/speechToText.ts -- not sure where this ought to live. I couldn't import from one to the other.

Addressed as a known duplication with comment explaining such in 197ec72

  • The model for LocalAISTT defaults to whisper if not provided - I assume this value comes from OpenAI's client.

SUGGESTIONS/QUESTIONS:

  • Should the placeholder be the default value when filling out a configuration? Or just serve as reminder text?
  • Specifcally for LocalAISTT, if a model isn't provided should it default to whisper-1 (the default model in the AIO packages of LocalAI)

Screenshots:

image
image
image

…a few debug statements around speech to text conversion. Finally, refactored the speechToTextProviders a bit to try and remove some magic strings that have undocumented rules around naming.
@clates
Copy link
Contributor Author

clates commented May 10, 2024

Thanks @HenryHengZJ - all feedback addressed in 197ec72

@HenryHengZJ
Copy link
Contributor

Thanks @HenryHengZJ - all feedback addressed in 197ec72

thanks! also verified openai and assemblyai works

@HenryHengZJ HenryHengZJ merged commit d3f03e3 into FlowiseAI:main May 13, 2024
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants